make the green backend the default on linux - #599
Merged
Conversation
green is faster on every shape this repo measures and the last blocker (dns holding a worker) landed in #598, so a spawned task now runs as a coroutine on the worker pool unless you say otherwise. the default stays os threads everywhere else. the reactor is epoll and eventfd, so macos and the bsds compile netpoll_fallback, which has no reactor at all: a green task waiting on a socket there would hold its worker for the whole wait. green is still reachable with PITH_GREEN=1 on those platforms, just not the default. PITH_GREEN now reads in both directions. 1/on/true force green, 0/off/false force os threads, and anything else takes the platform default. the off side is new and is the escape hatch for anyone the flip hurts, so it is trimmed and matched case-insensitively. an unrecognized value falls to the platform default rather than to os threads, so the only values that move the backend are the ones spelled out in the match and a typo behaves exactly like an unset variable. parsing moved into a pure backend_from_env so it can be tested without fighting the OnceLock that backend() caches into.
with green as the linux default, a comparison whose reference side just says `pith run` is green against green and passes for free. every green-* target now runs its os-thread side under PITH_GREEN=0. the same reasoning applies to coverage. `make test` on a linux box now exercises only green, so the PITH_GREEN=0 path across the corpus had no gate at all. verify-osthread-corpus mirrors verify-green-corpus for the opt-out and ci runs both. memcheck grew a small second list for the cases whose subject is the os-thread task machinery itself, so the flip does not quietly stop testing the code they were written for.
docs/concurrency.md drops the "off by default and still experimental" framing. the backend-choice section now says green on linux, os threads elsewhere, and PITH_GREEN=0 as the opt-out, and it keeps the caveats that came with the flip rather than burying them: file i/o has no yield point so a file read holds its worker and everything pinned to it; preemption is a build-time opt-in where the kernel gave os threads preemption for free; and task placement is first-resume luck. docs/limitations.md had a list of what it would take to make green the default. most of that is history now, so it reads as the caveats of the current default instead. the performance scoreboard labelled its concurrency rows "(green)"; the os-thread rows are the ones that need a label now, so they carry PITH_GREEN=0. the same for the bench README table and the examples and module headers that showed PITH_GREEN=1 as the way to opt in.
three small corrections on top of the flip. `PITH_GREEN=no` fell to the platform default, which on linux is green — so someone reaching for the escape hatch and writing the most natural word for "off" got exactly what they were trying to turn off. no/n now mean os threads and yes/y mean green, which keeps the rule (only spelled-out values move the backend) while making the direction that costs something hard to miss. the green corpus target claimed it "matches os-thread output". it compares against a fixed expected file, which is the stronger check and the reason it catches a drift both backends share. now that there is a sibling target that really does run os threads, the old wording is worth correcting. and the corpus is 261 cases, not 260, since the dns case joined it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
summary
green is now the default task backend on linux. os threads stay the default
everywhere else, and
PITH_GREENoverrides in both directions on any platform.backend()used to readPITH_GREENand treat everything that was not1/on/trueas os threads, which meant there was no way to say "off" asdistinct from "unset". that did not matter while os threads were the default; it
does now. the parsing moved into a pure
backend_from_envso it can be unittested without fighting the
OnceLockthatbackend()caches into, and it reads1/on/trueas green,0/off/falseas os threads, and anything else asthe platform default. values are trimmed and matched case-insensitively, because
the off switch is the escape hatch for anyone the flip hurts and should not be
missable by a capital letter.
unrecognized values fall to the platform default rather than to os threads. the
rule that comes out of that is short enough to hold in your head: the only values
that move the backend are the six spelled out in the match, and a typo like
PITH_GREEN=yesbehaves exactly like an unset variable. falling to os threadsinstead would have made a typo silently disable the platform default, which is
the more surprising of the two.
the default is linux-only because the reactor is.
netpoll.rsis epoll andeventfd behind
#[cfg(target_os = "linux")]; macos and the bsds compilenetpoll_fallback.rs, which has no reactor at all, so a green task waiting on asocket there holds its worker for the whole wait. green stays reachable on those
platforms with
PITH_GREEN=1, just not by default.what changed outside the runtime
a silent flip does most of its damage in the places that compared the two
backends by running one of them without saying so. every
green-*target in theMakefile ran its reference side as a bare
pith run; on a linux box that is nowthe green side, so the differentials were about to compare green against green
and pass for free. all twelve now run their os-thread side under
PITH_GREEN=0.make teston linux likewise exercises only green now, which left thePITH_GREEN=0path across the corpus with no gate at all.verify-osthread-corpusmirrors
verify-green-corpusfor the opt-out and ci runs both;verify-green-corpusitself was already explicit about its backend and stilltests what it says, with the added value that it holds on a mac too.
memcheck had the same hole in miniature:
test_os_thread_spawn_reclaimexists tovalgrind the os-thread slab reclaim path and would have started running green.
it and three other task-lifecycle cases now get a second pass under
PITH_GREEN=0, alongside the green pass they get from the default.bench/chan_fanout_bench.shalready setPITH_GREENexplicitly on both rows, soonly its prose and the
pithrow label needed to change.docs
docs/concurrency.mddrops the "off by default and still experimental" framing;the backend-choice section now describes green as the linux default with
PITH_GREEN=0as the opt-out.docs/limitations.mdhad a list of what it wouldtake to make green the default, which is mostly history, so it reads as the
caveats of the current default instead. the performance scoreboard labelled its
concurrency rows "(green)"; the os-thread rows are the ones that need a label
now.
the caveats stay documented in both places rather than buried. file i/o has no
yield point:
host_fsgoes straight to the syscall, so a file read holds itsworker and everything pinned to it, and a program can hit that purely by
upgrading. preemption is a build-time opt-in (
PITH_GREEN_PREEMPT=1) where thekernel gave os threads preemption for free. task placement is first-resume luck,
which is why the fan-out benchmark is bimodal at ~46ms pinned against ~130-170ms
split.
what was tested
cargo build --releaseat the repo root, thencargo test --release --workspace --locked: 115 tests pass, of which 98 are the runtime (up from 93, the six newbackend_from_envcases replacing the single old default assertion).make testunder the new default, on this linux box, so green: 99 examples, 261regressions, 3 invalid-parse, 37 invalid-checker, 10 cli, 6 ir-contract, then 261
self-hosted regressions, 3 self-hosted invalid-parse, 14 parity, 37 self-hosted
invalid-checker, 5 self-hosted cli. zero failures.
PITH_GREEN=0 make testproduces the identical counts, also with zero failures. nothing that passed under
os threads fails under green.
make green-tests(all eleven),make verify-green-corpus(261 cases at thedefault worker count and again at one worker), the new
make verify-osthread-corpus(261 cases),make run-examples(99),make docsite-check, andmake memcheck, which is valgrind-clean including thefour cases that now get a second os-thread pass.